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1* Introduction 

Scientific discovery follows a cyclic pattern of theory development - the formation of a new 
theory, a number of fixes to account for anomalous behavior and, later, when the fixes result in a 
grossly inelegant theory, a paradigm shift that radically transforms the old theory (Kuhn, 1970). 
Theory development is a multi-faceted process involving the synergistic cooperation of a number 
of approaches. This paper attempts to provide a comprehensive model for each stage of theory 
development by integrating several techniques - empirical learning, analogical learning, and 
experimentation* 

Our central task is to provide caus?l explanations of observed natural phenomena, using an 
imperfect or non-existent theory of the domain. This paper describes the integration of two 
techniques: verification-based analogical learning^ which is used primarily for theory formation 
(including paradigm shifts), and experimentation-based theory revision^ which is used primarily 
for theory testing and revision. (We review each only briefly here, for details see (Falkenhainer, 
1987) and (Rajamoney, 1988)). Analogy-driven theory formation can develop novel theories in a 
focused manner, drawing from its knowledge of analogous precedents. However, in isolation it 
lacks the ability to examine the state of the world and experimentally confirm or refute 
hypotheses. Experimentation-based theory revision, on the other hand, is well suited to 
modifying theories and testing the different modifications to a theory. However, it has very little 
guidance during the task of proposing different modifications to a theory. Alsc^ it must rely on 
fortuitous observations or prediction failures to initiate theory revision. 

In this paper, we describe a set of general principles underlying theory formation, theory 
revision, and experimentation. We demonstrate how these techniques, each focusing on different 
aspects of the problem, may be Integrated to provide a more unified and comprehensive 
treatment of scientific theory development. In particular, we show how experimentation is used 
to empirically guide and verify the analogy-driven theory formation stage and how analogy is 
used to initiate and focus the experimentation-based theory revision stage. Furthermore, we 
outline a general protocol for communication between theory formation and experimentation- 
based revision which solves many of their individual problems. Experimental queries using the 
protocol are motivated by empirical questions and requirements encountered during hypothesis 
generation and evaluation. Theoretical queries are motivated by the need for plausible hypotheses 
during experimentation-based theory revision. 

The utility and generality of this protocol is demonstrated by an implementation 
integrating two previously autonomous systems: PHINEAS, which performs verification-based 
analogical learning, and ADEPT, which performs experimentation-based theory revision. When 
confronted with a novel observation of evaporation, this integrated architecture and its 
associated protocol enabled PHINEAS to empirically question the necessity of heat flow (e.g., from 
a stove) in its evaporation hypothesis derived from boiling. In turn, the protocol enabled ADEPT 
to be advised of vapor saturation as a candidate explanation for why evaporation stopped. 

We conclude this section with a review of the two techniques. Xn integrated design and the 
general principles which emerge is then described in section 2 and made concrete with the use of 
an implemented example. An additional example is briefly provided in section 3. 
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1.1. VerincAtion-Baaed Analogical Leai-nings An Overview 

Verification-based analogical learning (VBAL) (Falkenhainer, 1987) is an approach to 
theory formation and revision which relies on analogical inference to hypothesize new theories, 
and gedanken experiments (i.e., simulation) to analyze their validity. VBAL is an iterative 
process consisting of four primary stages: 

!• Behavior match* An anomalous observation triggers a search for previously understood 
experiences that exhibited analogous behavior. The result of this stage is a candidate 
an;^iogue and an initial set of correspondences between the two domains.^ 

2. Theory generation. The central objective of the second stage is to produce a fully 
operational initial hypothesis about the current domain. This stage has two components. 
First, the model used to explain the recalled cxperienxe is analogically mapped into the 
current domain* This mapping is^guided by the initial correspondences found in the 
behavioral comparison. Second, the model must be operationalized if it is not already. For 
example, it may reference entities and properties that are currently unknown. These entities 
and properties must either be inferred from the domain theory or their existence must be 
postulated. 

3. Theory completion. The operational model may not necessarily conform to the present 
observation. Thus, theory revision may be required to examine predictions and produce a 
consistent initial model. Revision in VBAL is based on passive consistency - when empirical 
experiments are impossible, prefer minimal revisions with an analogical precedent. 

4. Theory revision. While the current domain model may accurately account for all known 
observations, a new situation may be observed for which the model is inadequate. 

PHINEAS, the current implementation of VBAL, uses Forbus' (1984, 1986) Qualitative Process 
theory to reoresent and reason about change in the physical world, and a modified version of 
Centner's \^38Z) Structure-^Mapping theory for analogical mapping. The system's primary 
analogy component, SME (Falkenhainer et al, 1986), is used to perform comparisons of similarity 
and map theories across domains. PHINEAS has been shown to dic^cover a caloric theory of heat 
by analogy to liquid flow. It has also developed an explanation of oscillatory electrical circuits 
from its knowledge of mechanical oscillation. 

Experimentation— Based Theory Revision: An Overview 

Experimentation-based theory revision (Rajamoney, 1988) is an approach to changing or 
augmenting an incomplete or incorrect theory. Experimentation-based theory revision consists 
of three main steps: 

1. Contradiction detection. Problems with the existing theory are detected by comparing 
the predictions based on the theory with the observations made from the real world. If the 
predictions and observations lead to a contradiction then the existing domain theory has to 
be revised. 



^ The term analogue is used in its most general sense tnroughout this paper. An anaiogy may be found withm the same domam, 
as betv:sen one instance of liquid flow and another, or across domains, as betv/een electron flow and liquid flow. No strong claims are 
made about the accus mechanism, whose theoretical underpinnings are still under development. While implemented and autonomous, 
the accb^ser is far from sophisticated at this time. 
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2. Hypothesis formation* Hypotheses that involve changes to the existing theory are 
proposed. This stage is governed by explanation construction - only changes leading to a 
revised domain theory that can explain the observed phenomenon are proposed. 

3. Experiment design. Typically, there will be a number of different ways to change a theory 
to explain a new phenomenon. Experiments are designed to identify the change that is 
consistent with the actual behavior of the real world (see Rajamoney, 1988). Hypotheses 
whose predictions are not consistent with experimental observations are rejected. 

ADEPT is a system that demonstrates experimentation-based theory revision. It starts with an 
operational but initially incomplete and incorrect theory and revises it to conform to 
observations made from the real world. ADEPT has been demonstrated on a number of examples. 
It has successfully learned effects and conditions of physical processes such as a new influence of 
evaporation on the temperature of the evaporating liquid and a new condition for dissolving 
solutes requiring the concentration of the solution to be less than its saturation concentration. 

2. An Integrated Model of Theory Development 

Theory development is a djaamic process involving the formation of candidate thaories, 
revisions to account for anomalous observations or achieve simplicity, and experimentation to 
test hypotheses and collect new data. These activities are highly interdependent yet highly 
modular. They may be performed by numerous people in distant locations, possibly across 
generations (e.g., the discourse between Copernicus, Galileo and Kepler, whose lives spanned 
nearly 200 years). In this section we develop this "collection of specialized experts" view of theory 
development with respect to theory formation, revision, and experimentation and show how the 
expertise of each is required to reduce the search space throughout the development cycle. We 
first describe strengths and limitations of our two methodologies and their complementary 
relationship. We then define a general protocol for interaction between systems specializing in 
theory formation, revision, and experimentation. Next, we show how the integrated system 
provides a comprehensive model for each stage of theory development. 

Limitations and Strengths 

As independent research efforts, each approach is designed to perform theory development 
autonomously. VBAL conjectures explanations of the new behavior based on its similarity to 
previously encountered phenomena. Its power lies in its ability to construct novel theories, 
focused by considering only theories for which it has an analogical precedent. However, VBAL 
has some important limitations. It is intended for passive observation and thus does not 
empirically test a hypothesis. It makes intermediate assumptions and depends on simulation 
experiments to test the consistency of these assumptions. This leads to questions about the 
validity of proposed theories. In addition, VBAL must rely on heuristics and prior experiences to 
guide search, thus requiring one to entertain hypotheses that might easily be empirically refuted. 

In turn, experimentation-based theory revision locates known theories that appear relevant 
to the observation and modifies them to inclusively account for the new example. Its power lies in 
its ability to interact actively with the world, discriminate among alternative revisions, and 
refute proposed hypotheses. However, experimentation-based theory revision has some important 
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limitations. It is unable to make large, fundamental modifications to a theory, or produce one 
where none existed before. This limits it to revising "roughly correct'* theories. Furthermore, 
experimentation is essentially open-ended in terms of what tests can be made and what kinds of 
experiments can be designed. For example, it may not be tractable to collect all possible 
hypotheses prior to conducting experiments* 

The limitations of each approach corresponds nicely to the strengths of the other. 
Verification-based analogical learning provides: 

L Highly focused, theoretically motivated queries about the state of the world, such as 
checking for the existence of an anticipated object or testing theoretical predictions. 

2* Large changes in theoretical perspective, develops new theories and provides candidate 
theories Jbo revise. 

3. A restricted number of potential hypotheses and a preferential ordering on hypotheses. 
Experimentation-based theory revision pro /ides: 

1. Empirical tests for the relevance of certain perceived aspects of a situation, such as the 
necessity of ancillary processes normally associated with an analogue theory. 

2. Discrimination among plausible hypotheses or refutation of a single hypothesis. 

3* Experiments to detect the presence of an unnoticed or conjectured object or tests for 
conjectured properties of existing objects* 

4. Probes of the situation for unknown values of a quantity or its derivatives* 

2*2* A Communication Protocol 

In this section we develop a general protocol for theory development tasks which enables 
one method to make use of another's expertise in a system independent manner. If the process 
answering a query lacks the appropriate decision procedure or is unable to successfully produce a 
response, a value of UNKNOWN is always acceptable. The protocol results in an extendible 
architecture, allowing new techniques to be added or existing techniques to be replaced by 
improved versions. 

A crucial question concerns how one tells whether or not some fact holds. In theory 
formation and revision, we must assume incomplete knowledge* Thus, normal ways of deriving 
facts about the environment must be redefined. We distinguish bctv;een queries which depend 
primarily on the current beliefs of the system and queries which are able to question those beliefs 
' by looking for alternate sources of information* Direct queries directly probe for the status of a 
condition using inference, or simple empirical tests if their status is not easily deducible* Indirect 
queries look for the effects normally associated with a condition, to test if deductive conclusions 
are wrong or if an analogous condition is in effect. 

We decompose theory development into eight tasks, which suffice for a wide range of 
interesting problems: 

Present? <object-or-variable > < primary-conditions > <3econdary-conditions > 

This queries for the existence of a partially specified, unknown object. Failure of any 
secondary conditions is not grounds for rejection, but may be used to reduce the set of 
candidates satisfying primary conditions. For example, investigation of a chemical reaction 
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may lead one to suspect and test for the presence of a particular catalyst in the mixture. 

Test-Value < quantity > < value > 

This instructs the system to search for an inferential determination of the quantity's value, 
or experimentally probe the situation for the value. The value may be left unspecified, or 
the truth of a particular relative value may be sought, as in (GREATER-THAN boiling- 
temperature) or DECREASING, For example, hypothesis generation may require the value of 
a quantity act reported in the initial observation* . 

Test-Condition <relatlon> 

This determines if th:» specified relation holds. For example, investigation of heat-related 
theories may require determining if one object is in thermal contact with another. 

Test-Effects < relation > 

This is an indirect query which instructs the experimentation system to look for the 
presence of the known effects of a physical relation. For example, if Test-Condition had 
determined that a specified fluid path was not configured to support liquid flow, the validity 
of this conclusion may be questioned by looking for the effects of liquid flow through the 
path. Test-Effects may be used to seek an answer when Test-Condition returns 
unknown, instigate belief revision, or create a new concept analogous to the given physical 
relation. 

Necessary? < condition > < observation > 

The necessity of a condition thought to be essential to achieving an observation is 
questioned with this query. For example, a developing theory on chemical reactions may 
state that a particular catalyst is required. This query would call for experiments to 
determine if the catalyst is actually required; 

Discriminate < hypotheses > < observation > 

This asks the experiment design system to construct experiments to discriminate among a 
set of hypotheses that are thought to cause the observation. For example, during theory 
revision, several modifications of the original theory may explain the novel observation. This 
query finds those revisions that are consistent with information gathered from directed 
experiments. 

Propose-Theor ies < observation > 

This invokes the theory formation system to develop theories that will explain the 
observation. For example, during theory revision, experiments may rule out all the candidate 
revisions or the revision proposer may not find any consistent modifications to the theory. In 
such cases, the theory revision system uses this query to post a theory formation task. 

Propose-Revisions <observation> <thcory> <:condition or :efT€r.t> 

This query asks the theory formation system, or a specialist in revision hypotheses, to 
propose revisions to a theory that will enable it to explain the observation. The query must 
specify if the revision should be to the theory's conditioning relations or to its set of effects. 

2.3. The Three Stages to Developing a Complete and Consistent Theory 

There are three primary stages to theory development within the VBAL framework: 
Theory Generation, Theory Completion^ and Theory Revision. This section examines the 
interaction of the two systems in each of these stages and uses an implemented example to 
illustrate the interaction. The example involves forming and revising a model for evaporation. 
Throughout this section, we will assume that theories are represented by a set of processes and 
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individuaU (objects) as defined in Forbus' (1984) Qualitative Process theory. Each process 
specifies a set of participatory individuals (objects and other processes), a set of conditioning 
relations indicating when it is active, and a set of effects that apply when it is active. 

In our implementation of this integrated architecture, PHINEAS initiates the process of 
explaining new observations and posts tasks for ADEPT when its expertise is required. If an 
observation cannot be explained by existing theories, PHINEAS attempts to generate za 
explanation by recalling past observations of similar or analogous behavior. The knowledge used 
to explain these previous observations is used to analogously explain the current situation. In 
constructing the explanation, PHINEAS may call on ADEPT to measure the value of a quantity or 
cLetermine the existence of a hypothesized object. In addition, iixperimisntation and possible 
revision of the initial hypothesis may then be required for this or future observations. PHINEAS 
and ADEPT use the protocol to interactively solve such problems during theory formation and 
revision. 

Theory Generation 

Theory generation is the process of proposing an initial theory to explain a novel 
phenomenon. There are two stages to theory generation in VBAL. The first stage examirxes the 
current behavior and recalls pr^^viously understood similar behavior. Theories used to explain ai 
analogue behavior are then mapped into the current domain and proposed as an initial 
explanation of the current phenomenon. 

The system next divides the elements of each hypothesized theory into primary processes 
and ancillary processes, A primary process is one that affects an observed quantity, such as the 
temperature of an object whose temperature was observed. Ancillary processes are those that 
were used in explaining the analogue case, but do not directly affect one of the current 
observables. The necessity of a primary process is assumed and never experimentally questioned. 
Ancillary processes are eliminated during theory formation if they are found to be unnecessary. 

The second stage in theory generation is tc make the proposed explanations operational. A 
model is non-operational if it calls for participatory objects and processes whose presence was 
not noticed during the initial observation. If their existence cannot be determined (at this time), 
their existence must be assumed or the proposal modified co not depend on them. 
Operationalization first consists of instantiating each primary process by examining its unknown 
individuals: 

Wken the individual is an uninstantiated, ancillary process: 

Attempt to instantiate it. 

If not instantiable, test for NECESSARY?. 

If necessary, reclassify it as a primary process, otherwise remove it. 
When the unknown individual is an entity: 

Test for PRESENT? 

If not, postulate its existence and store its known set of characteristics. 

Any ancillary process that is experimentally determined to be irrelevant is delettid from the 
generated theory. Remaining ancillary processes must be instantiated, if they have not been 
already. However, if that instantiation calls for postulating the existence of an unknown entity, 
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(Hecessary? <ucillar7*-proce88> <ob8ervatlon>) will be invoked first. If a later 
contradiction occurs, the ancillary processes should be the first to be questioned during revision. 

The final step in producing a fully operational theory is to test the analogically proposed 
conditioning relations for validity undei' the current conditions. This is done using Test-Value 
and Test-Condition. Conditioning relations not known to be true in the current context are 
temporarily removed in order to obtain a fully operational theory and enable discrimination with 
competing theories* For surviving theories, an attempt is then made to replace these 
conditioning relations with analogous relations applicable to the c« Trent context. 

2«3«X«1« Evaporation Ebcample: Generating Initial Hypotheses 

This section describes how the integrated implementation generates an initial model for 
evaporation. The system begins with knowledge of six processes - liquid flow, heat flow, 
dissolving, spring oscillation, boiling and heat replenishment (to constantly maintain the heat of 
a stove). It also has a database of physical observations fully explained by these processes. 
Suppose an open beaker containing alcohol is left on a table top for a day, during which time the 
amount of alcohol is observed to decrease* The problem is to propose an explanation for this 
observation, which, according to the system's existing knowledge, should never have happened. 

Model Creation: At this point, no known, instantiable theory is able to even partially 
explain the loss of alcohol in the beaker. By examining its knowledge of ''similar'' behaviors, 
PHINEAS determines that the only known examples of a liquid leaving a container call for it to 
flow out or to boil away. Three ordered, initial hypotheses, based on overall behavioral 
similarity, are generated. The first calls for the alcohol to flow out, the second calls for it to 
vaporize, and a distant third calls for it to dissolve based on its gradual disappearance. The 
initial vaporization explanation is shown in Figure 1. It consists of one primary process - boiling, 
and two ancillary processes - heat flow and keat replenishment, which are believed to be required 
for boiling to take place. 

Model OperaHonalization: The three models (based on flow, boilmg, and dissolving) call for 

the presence of objects that were undetected in the initial observation. We focus first on 

operationalizing the vaporization explanation. This explanation requires the presence of two 

unknowns - alcohol vapor in the beaker and a heat source. PHINEAS first produces the query: 

(Present? 7STEAH1 ( (Contalncd-Gas 7STEAM1) /primary conditions 

(Contalner-of 7STEAM1 beaker) 
(Substance-of 7STEAM1 alcohol)) 
()) ;no secondary conditions 

Notice that the query is very specific - the object is fully specified and all that remains is to test 
if it is there. Of course, even this test can be non-trivial. To answer the query, ADEPT first 
examines the known set of objects and finds that none satisfy the primary conditions. It then 
examines its database of experimentation techniques, and recalls that litmus paper changes color 
in the presence of alcohol and can be used to test for the hypothesized presence of alcohol vapor 
in the beaker. The litmus paper test is conducted, via instructions to the assumed human 
assistant (such as, ''make the litmus paper touch the air inside the beaker"), and a positive result 
confirming the presence of alcohol vapor is obtained. 
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ACCESSr Found the foUovlng applicable to observation OBSERVATION-r 
3MAP(2-C0NTAINER-LF.0BSEaVATl0N-l) 
BMAP(BQILING-PROTOTYPE.OBSERVATION-I) 
BKAP (SALT-DISSOLVING, OBSERVATIOH-l) 

0 0 0 

(B-EXPUINS 

(SET (PROCESS-DEFINITION PIl ;itri9€i from ht^i flow, A% €%ciU€Tf proctu. 

(IMPLIES 

(AND (INDIVIOUAL fSTOVS (CONDITIONS (QUAtrnTY (HEAT fSTOVE)))) 

(INDIVIDUAL ALCOHOLl (CONDITIONS (QUANTI"^ (HEAT ALCOHOL!)))) 
(INDIVIDUAL BEAKERl (CONDITIONS (HEAT-PAfH 3EAKERI) 

(HEAT-CONNECTIGN BEAKERl WTOVK ALCOHOLl))) 

(HEAT-ALIGNED BEAKERl) 

(GREATER-THAN (A (TEMPERATURE 9ST0VE)) (A (TEMPERATURE ALCOHOLl)))) 
(AND (QUANIITY (HEaF-FLOW-RATE PIl)) 

(Q= (HEAT-FLOW-RATE PIl) (- (TEMPERATURE fSTOVE) (TEMPERATURE ALCOHOLl))) 
(I^ (HEAT ALCOHOLl) (A (HEAT-FLOW-RATE PIl))) 
(I- (HEAT ?STOVE> (A (HEAT-R IW-RATE PIl)))))) 
(PROCESS-DEFINITION PI2 litriwi from boiling. Tht frirrury procett. 

(IMPLIES 

(AND (INDIVIDUAL ALCOHOL (CONDITIONS (SUBSTANCE ALCOHOL))) 

(INDIVIOUAL BEAKERl (CONDITIONS (CAN-CONTAIN BEAKERl ALCOHOL))) 

(INDIVIDUAL ALCOHOLl (CONDITIONS (CONTAINED-LIQUID ALCOHOLl) 

(CONTAINER-OF ALCOHOLl BEAKERl) 
(SUBSTANCE-OF ALCOHOLl ALCHOHOL))) 

(INDIVIDUAL fSTEAMi (CONDITIONS (CONTAINED-GAS fSTEAMl) 

(CONTAINER-OF tSTEAMt BEAKERl) 
(SUBSTANCE-OF fSTEAMl ALCHOHOL))) 

(INDIVIOUAL PIl (CONDITIONS (PROCESS- INSTANCE HEAT-FLOW PIl) 

(DESTINATION PIl ALCOHOLl))) 

(ACTIVE PIl) 

(NOT (LESS-THAN (A (TEMPERATURE ALCOHOLl)) (A (TBOIL ALCOHOLl)))) 
(GREATER-THAN (A (AMOUNT-OF ALCOHOLl)) ZERO)) 
(AND (QUANTITY (VAPORIZATION-RATE PI2)) 

(Q= (VAPORIZATION-RATE PI2) (HEAT -FLOW-RATE PIl)) 
(GREATER-THAN (A (VAPORIZATION-RATE PI?)) ZERO) 
(I- (HEAT ALCOHOLl) (A (VAPORIZATION-RATE PI2))) 
(I- (AMOUNT-OF ALCOHOLl) (A (VAPORIZATION-RATE PI2>)) 
(I^ (AMOUNT-OF 7STEAM1)) (A (VAPORIZATION-RATE PI2))))) 
(PROCESS -DEFINITION PIS ;ieritei from httt repttnCtimtnl. An aneilUry procea, 

(IMPLIES 

(AND (INDIVIDUAL 9ST0VE (CONDITIONS (HEAT-SOURCE f STOVE))) 

(INDIVIOUAL PIl (CONDITIONS (PROCESS-INSTANCE HEAT-FLOW PIl) (SOURCE PIl fSTOVE))) 

(ACTIVE PID) 
(A!^D (EQUAL-TO (D (HEAT 7ST0VE)) ZERO) 

(I^ (HEAT fSTOVE) (A (HEAT -FLOW-RATE PIl))))))) 
OBSERVATION- I) 

Figure 1: Creating an initial explanation of the evaporation observation. Here, PHINEAS hy- 
pothesizes, via behavioral analogy, -that the set of three boiling-related processes explains the 
evaporation observation. At this stage, these processes are non-operationaL 



PHn^AS next produces a Present? query for the heat source. In this case, ADEPT returns 
a value of unknown: unable to deductively locate such an object or experimentally test for one. 
PHINEAS responds with 
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(Hecessary? ?I1 OBSERVATION-l) 

which asks if the ancillary process PU (heat flow) is necessary to produce the observation. 
ADEPT proposes to repeat the observation, only this time thermally isolating the alcohol from 
potential heat flows by using an insulated container* When the amount of alcohol still decreases, 
the need for an external supply of heat is removed since it must be irrelevant to a potential 
explanation* 

At this point, the vaporization explanation is fully instantiated and consists of a single, 
vaporisation process* To make it fully operational, PHINEAS concludes with the following two 
queries about its proposed conditioning relations: 

(Test-Value (A (Temperature alcohol!)) -i(Less-Tlian (A (TBoll alcoholl)))) 

(Test-Value (A (Anount-of alcoholl)) (Grcater-Than zero)) 

ADEPT determines that the first condition is false and the second is true* PHINEAS then deletes 
the first condition and retains the second condition in the vaporization model*^ At t^is stage, the 
newly-formed vaporization model is completely instantiated, operational, and can iv^^y explain 
the observed decrease in the amount of alcohol in the beaker* 

The experimentally determined answers to these theoretically-motivated questions enabled 
the system to (1) confirm that alcohol vapor is present, (2) find that no known heat source exists, 
(3) determine that the type of heat flow associated with boiling is not a necessary component of 
the newly generated rocdel of evaporation and (4) find that the boiling temperature is not 
required for the new process* A new evaporation model is conjectured as the explanation* The 
alternative derived from liquid flow is rejected due to the need to hypothesize the existence of 
unobservable entities (e*g., a destination liquid), which was not required of the evaporation model 
(i.e*, Occam's razor)*^ 

2*3*2* Theory Completion 

Once an operational model has been produced, it must be tested for consistency with the 
observed behavior by using it to envision (Forbus, 1984) the possible behaviors of the current 
physical configuration* This corresponds to asking the question "what are the consequences of 
assuming this new theory?" Simulation may produce unanticipated theoretical predictions or 
uncover unanticipated iateractions with existing theories (Falkenhainer, 1987)* Experimentally 
testing theoretical predictions enables the system to refute a given hypothesis or strengthen its 
credibility* When unanticipated interactions among theories nullify the anticipated consequences 
of a hypothesized model, these interactions must be analyzed and theories revised* 

At the close of theory completion, a (potentially empty) set of hypothesized models exists 
that completely and consistently explain the observed behavior* If more than one hypothesis is 
produced, the DISCRIMINATE task is used to eliminate those that can be experimentally refuted* 

^ At model selection time, the attempt to restore the temperature condition fails when no analogous condition can be found. 

' Once the need arose to hypothesise unseen entities, PHINEAS's agenda mechanism temporarily abandoned the liquid flow ap- 
proach in favor of the boiling approach. The success of the boiling approach caused the system to permanently abandon the parcially 
complete liquid flow hypothesis. Being third in preference, the dissolving hypothesis was never examined. 
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2*3»2»1» Evaporation Example: Completing the Theory Formation Process 

The operational vaporization hypothesis contains the following three influences: 

(I- (Heat alcoholl) (A (Vaporization-Rate PI2))) 

(I- (Amount-of alcoholl) (A (Vaporization-Ra:e PI2))) 

(1+ (Amount-of alcohol-vaporl) (A (Vaporization-Hate PI2))) 

The influences on amount are consistent with the initial observation that the amount of alcohol 
liqilid decreased* However, when this model is used to anticipate what will happen to a beaker of 
alcohol left sitting on a table, it produces two secondary predictions - the alcohol's temperature 
will drop, due to the loss of latent heat during vaporization, and the amount of vapor will 
increase. Since the alcohol's teniperature across time was not originally reported, Test-Value is 
invoked for the anticipated change in temperature. ADEPT calls for the physical scenario to be 
repeated and changes in alcohol temperature noted. This test confirms the evaporation theory's 
hypothesis and verifies it as a complete and consistent explanation for the observed phenomenon. 

2»3*3» Theory Revision 

Theory revision occurs when an anomalous observation violates an existing model. The first 
step to revising an imperfect theory is blame assignment, which is done in a layered manner. 
First, hypotheses are made to determine if the contradiction h due to a process being incorrectly 
active or inactive, or if a process has an inappropriate causal efTect. Once candidate processes 
have been identified, specific revision hypotheses are generated to modify either the process' 
conditions or its causal effects. Experimentation is used at each stage to eliminate refutable 
hypotheses. Experiments are proposed until the correct revision to the theory is found or until 
all hypotheses have been eliminated. This last case calls for a new round of theory formation. 

2.3*3*1* Evaporation Example: Accounting for New Anomalous Observations 

The newly formed theory of evaporation predicts that it will always be active as long as 
there is liquid in a container. When this theory is used to explain a second example of 
evaporation - evaporation in a closed container - a contradiction is detected when the amount of 
liquid stops decreasing after only a little liquid has disappeared. Since the system recognizes this 
as an instance of evaporation (which stopped too soon), ADEPT is invoked to revise the current 
model. It first determines experimentally that the problem is due to a failed conditioning relation 
rather than a failed effect relation. It thus queries PHINEAS (using PRQPOSE-REVISIONS) for a set 
of candidate conditioning revisions. PHINEAS again recalls liquid flow and dissolving, due to their 
behavioral similarity and on the grounds that they were both observed to stop after being active 
for some time (i.e., the relevant problem with evaporation). Through analogous explanations for 
stopping, it proposes two new candidate quantity conditions - the amount of liquid has to be 
greater than the amount of vapor (from liquid flow) or the amount of vapor has to be less than 
soiue saturation point for the vapor (from dissolving). ADEPT experimentally refutes the first (by 
constructing an experiment to increase the amount of liquid and observing that evaporation does 
not start again) and finds the second to be consistent with experiments. This new saturation 
quantity condition is added to the evaporation model, yielding a theory consistent with existing 
experiences (Figure 2). 
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(PROCECS-3318 

Individuals ((?V-3309 (Substance ?V-3309)) 

(?V-3310 (Can-Contain ?V-3310 ?V-3309)) 
(?V-331l (Contained-Llquid ?V-33ll) 

(Containnr-of ?V-3311 ?V-3310) 
(Substauce-of ?V'-3311 ?V-3309)) 
(?V-3312 (Contain«d-Gas ?V-3312) 

(Container-of ?V-3312 ?V-3310) 
(Substance-of YV-3312 ?V-3309))) 
QuantityCondlvions ((Greater-Tlian (A (Amount-of ?V-3311)) zero) 

(Less-Than (A (Amount-of ?V-3312)) (A (Saturation-3326 ?V-3312)))) 
Relations ((Quantity (Vaporization-Rate ?self)) 

(Create r-Than (A (Vaporization-Rate ?self)) zero)) 
Influences ((I- (Heat ?V-3311) (A (Vaporlzatlon-Rate ?self))) 

(I- (Aoount-of ?V-3311) (A (Vaporization-Rate ?self))) 
(1+ (Aoount-of ?V-33J2) (A (Vaporlzatlon-Rate ?self))))) 

Figure 2: The hypothesized model of evaporation in Its final, revised form. 



3. An Additional Example: Understanding Osmosis 

Consider a case of forming and revising a model for osmosis. In this example, the system is 
shown two containers separated by a partition which, unknown to the system, is semi-permeable 
(Figure 3). It observes the amount of solution decreasing in one container and increasing in the 
other". The system is unable to explain these observations since none of the known processes are 
active. A number of processes are candidates for revision: absorption, boiling, condensation, 
evaporation, or flow. However, due to its ability to detect behavioral similarity, the system 
focuses first on liquid flow. Using the Present? query, it finds two potential paths for a flow: the 
membrane separating the two solutions and the wall of fcheir shared container. PHINEAS then 
finds, using Test-Condition, that neither object b Fluid-Aligned (able to transport fluids), 
since they are solid objects. It temporarily removes this precondition to form two operational, 
competing theories. ADEPT, in response to a Discriminate task, experimentally rules out flow 
through the container wall while all experiments continue to substantiate the membrane 
hypothesis. Now that it has a complete and consistent theory, PHINEAS looks for an analogue to 
the Fluid-Aligned precondition. After positively testing for the effects of the more general 
aligned concept, PHINEAS hypothesizes an analogue to Fluid-Aligned (called Aligned-34i7)j 
which ADEPT determines is a function of the membrane substance. The dependency of osmosis 
on concentration may then be determined during revision to accommodate future observations 
(as in Rajamoney, 1986). 

4. Evidence from the History of Science and Psychology 

History is filled with examples of analogy being used for hypothesis generation, followed by 
systematic analysis and experimentation (e.g., Oppenheimer, 1956; Dreistadt, 1968; Centner & 
Jeziorski, 1987). For example. Black's theories of latent heats of vaporization were developed by 
analogy with previously theories of melting. A year later, when a steady heat source was 
available, he conducted qaantitative experiments (Roller, 1961). One particularly clear example 
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Figure 3: Osmosis example. The solution level in the left chamber is decreasing, while the solu- 
tion in the right chamber is increasing. 



of analogical hypothesis generation appears in Carnot's use of an elaborate analogy between 
water-driven and heat-driven engines in his development of the Carnot cycle: 

In the waterfall the motive power is exactly proportional to the difference of level between the higher and 
lower reservoirs. In the fall of caloric the motive power undoubtedly increases with the difference of tem- 
perature between the warm and the cold bodies; but we do not know whether it is proportional to this 
difference... It is a question which we propose to examine hereafter. (Carnot, 1977, page 15) 

This hypothesis was later e3cperimentally verified and may now be derived from the first law of 
thermodynamics. While Carnot assumed the caloric theory, the Carnot cycle and Carnot 
principle laid the foundations for the second law of thermodynamics, which is independent of 
caloric or mechanical theories of heat. The psychological literature also includes numerous 
studies of people using analogy to hypothesize solutions to problems or explanations of physical 
behavior. Construction of mental or physical experiments is a commonly observed post- 
hypothesis behavior (e.g., Clement, 1986; Collins & Centner, 1987). 

5. Related Work 

Unlike the data-driven, weak methods of traditional systems in machine discovery such as 
BACON, STAHL, DALTON (Langley et al, 1981, 1987) and ABACUS (Falkenhainer et al, 1986), 
our system is theory-driven and knowledge-intensive. More closely related to our integrated 
model of theory formation, revision, and experimentation are Dietterich and Buchanan's (1983) 
EG and Shrager's (1987) IE. EG stressed the need for experimentation in theory formation and 
focused on hs>w experimentation could constrain a potentially unwieldy hypothesis generator. 
While we incorporate this aspect, wc also emphasize the important role that a highly focused 
theory generator may have on experimentation. In IE, the central claim is that careful causal 
analysis is unnecessary for tasks such as forming simple models of complex devices. Our primary 
concern is scientific theory formation which calls for a more methodical approach, though it 
seems that our design could apply to understanding everyday devices. Our work also shares some 
similarities with the IDS system (Nordhausen & Langley, 1987), which uses experimentation to 
induce qualitative models. Unlike IDS, our system uses analogy to focuu the formation and 
revision of th^iories, and directed experiments to test the generated and revised theories. 
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6* Discussion 

Here we have explored the strengths of combining two distinct theory formation and 
•revision systems to form an integrated system based on analogy and experimentation. We 
showed how such an integration results in a more complete account of the entire scientific 
discovery process. Theory formation, revision and experimentation are viewed as interrelated and 
interdependent processes that require each system to postulate well-defined, theoretically 
motivated queries to use the expertise of the other. 

The evaporation example illustrates many finer points of the discovery process. In the 
evaporation example, a person typically wouldn't generate all possible hypotheses (e.g., 
evaporation, liquid flow, absorption, vanish, invisible, etc.) and then discriminate between them. 
The space of hypotheses is made tractable by pruning based on a knowledge of what can happen 
and a knowledge of what is likely to happen (based on analogical precedents). Also, the presence 
of an experiment generator enabled us to test for the presence of alcohol vapor, rather than 
having to assume its existence and expend effort.on a potentially incorrect hypothesis. 

We are still far from programs that truly reflect actual scientific discovery processes. For 
example, we are -Jtrongly bounded by the complexity of qualitative reasoning and must limit 
explorations to discovering overly simplistic theories. Furthermore, the knowle( e bases and 
reasoning sophistication of existing discovery systems are far too small to escape the "discovering 
Newton' laws in a day" phenomenon. However, this work takes steps towards realizing the scale 
and flavor of the task. Furthermore, it has made limitations in the systems clearer and provides 
a catalyst for further development. Future research in this direction involves modeling an entire 
cycle of scientific discovery using an example from the history of science such as the formation, 
revision and, finally, rejection of the caloric theory of heat. 
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